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(57) Abstract 

A method and system (10) for detecting vehicle occupant type and position utilizes a single camera unit (12) positioned, for example 
at the driver or passenger side A-pillar, to generate image data of the front seating area of the vehicle. The present invention distinguishes 
between objects, forwardly or rearwardly facing child seats, and occupants, by periodically mapping the image taken of the interior of the 
vehicle into an image profile (104), and utilizing image profile matching with stored profile data (1 10) to determine the occupant or object 
type. The system and method of the present invention track occupant type and position in both parallel and perpendicular directions relative 
to a fixed structure such as the vehicle instrument panel to optimize both the efficiency and safety in controlling deployment of a occupant 
safety device, such as an air bag (28). 
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OCCOPANT TYPE AND POSITION DETECTION SYSTEM 



BACKGROUND OF THE INVENTION 
The present invention relates generally to motor 
vehicle crash discrimination systems utilized for actuating or 
5 deploying a passenger safety restraint, and more specifically 
to a system and method for detecting occupant seating 
conditions so as to optimize deployment of a passenger safety 
restraint , 

Conventional vehicle crash discrimination systems 

10 typically employ at least one mechanical, electromechanical, or 
electronic acceleration sensor affixed to the vehicle for 
sensing vehicle acceleration. The output of the sensors are 
supplied to a discrimination circuit for comparison to a 
predetermined threshold value. If the predetermined threshold 

15 value is exceeded, the discrimination circuit will output a 
signal which actuates or deploys a passenger safety restraint, 
such as an air bag or passive seat belt mechanism. 

However, conventional mechanical or electromechanical 
accelerometer based crash discrimination systems do not account 

20 for variations in passenger/ occupant conditions in determining 
whether to actuate the safety restraint. More specifically, 
conventional accelerometer based crash discrimination systems 
are generally designed to assume nominal conditions, such 
as 50th percentile male, actual presence of a vehicle occupant, 

25 and failure of an occupant to wear a seat belt. The assunption 
of these crash conditions are necessary to insure proper 
actuation of the safety restraint when severe deceleration of 
the vehicle is detected by the accelerometer. Such assumptions 
inherently cause unnecessary, undesired, or improperly- timed 

30 actuation of the safety restraint in conditions where no 
occupant is present, in marginal crash situations where a seat 
belt provides sufficient safety protection for the occupant, or 
in situations where the occupant is improperly positioned 
relative to the safety restraint such that actuation of the 

35 safety restraint could potentially injure the occupant. 
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Thus, since conventional crash discrimination systems 
can not accommodate various occupant conditions which affect 
the desirability of actuating the safety restraint, they have 
not proven to be completely satisfactory. In response, the 
5 prior art has attempted to overcome these deficiencies by 
providing arrangements which are generally directed at 
detecting occupant presence, size, or position relative to some 
fixed structure in the vehicle. The following are examples of 
such prior art arrangements. 

10 US 5,413,378 to Steffens, Jr., et al disclose a 

system for controlling an occupant restraint, such as an air 
bag, wherein the system utilizes a combination of a set of 
ultrasonic occupant position sensors, and various seat and 
occupant weight sensors, to determine occupant weight and 

15 position relative to fixed structure with the vehicle. 

US 5,398,185 to Omura discloses a system for 
optimizing deployment of passenger restraint devices which 
utilizes a combination of a plurality of seat sensors, a card 
reader for inputting data regarding the physical 

20 characteristics of the occupant, and two telecameras to compute 
a value characteristic of each interior vehicle element and the 
occupant's estimated behavior relative thereto. 

US 5,366,241 to Kithil discloses an overhead -mounted 
air bag deployment system which utilizes an overhead passenger 

25 sensor array to sense position and velocity of an occupant's 
head so as to control deployment of an air bag, and to detect 
and provide warning when the occupant is in an unsafe seated 
condition. 

US 5,074,583 to Fujita et al disclose a vehicle 
30 collision detection system which utilizes a plurality of seat- 
mounted sensors to detect occupant seating condition, position, 
and size in order to optimize inflation of an air bag in a 
vehicle collision. 

In addition, commonly owned U.S. Patents 5,446,661 
35 and 5,490,069 each disclose a method and system for vehicle 
crash discrimination which continuously detects various vehicle 
occupant positions for optimizing a discrimination analysis to 
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achieve increased efficiency and reliability in actuating a 
safety restraint. 

While these arrangements may have provided an 
improvement in efficiency over conventional crash 
5 discrimination systems, there still exits a need for a crash 
discrimination system which can further optimize or tailor air 
bag deployment based on the specific type of occupant present 
in the vehicle. More specifically, with the increased use and 
availability of air bags in motor vehicles has come the 
10 realization that deployment of an air bag in certain crash 
situations, and with certain types of occupants, such as 
infants strapped into a child safety seat, has the potential of 
causing more harm to the occupant than if the air bag were not 
deployed. 

15 As noted above, this problem has become particularly 

acute with infant safety seats. The prior art has attempted to 
distinguish passengers from infant child seats by using 
conventional distance measuring techniques to detect the amount 
and extent of possible occupant movement, or alternatively has 

20 used weight sensing arrangements to detect the weight of any 
object which might be located on the vehicle seat. In either 
arrangement, threshold values are used to classify an object as 
either a passenger or an inanimate object. 

However, simply using weight sensors or movement 

25 monitoring has not provided the level of discrimination between 
occupant types or the reliability necessary to achieving 
effective "smart" control over air bag deployment. As a 
result, a need still exist for a system which can automatically 
and continually determine occupant type and position in a 

30 reliable and cost effect manner. 

SUMMARY OF THE INVENTION 
It is therefore an object of the present invention to 
provide a system and method which automatically determines the 
type of occupant in a vehicle seat, as well as the location of 

35 that occupant within the seat relative to fixed structure in 
the interior of the vehicle, such as the dashboard or steering 
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wheel, so as to increase efficiency and reliability in 
actuating or deploying a safety restraint such as an air bag. 

It is another object of the present invention to 
provide a system and method which is capable of detecting the 
5 presence of either a person, a rearward facing infant car seat, 
a forward facing infant car seat, or a box or other inanimate 
object, so as to increase efficiency and reliability in 
actuating or deploying a safety restraint such as an air bag. . 

It is a further object of the present invention to 

10 provide a system and method which determines if an occupant is 
in an unsafe seated position to optimize control over 
deployment of an occupant safety restraint such as an air bag 
or other passive restraint device. 

In accordance with these and other objects, the 

15 present invention provides a system and method which detects 
occupant position and type which utilizes a single camera unit 
positioned for example at the driver or passenger side A- 
pillar. The present invention provides a system and method 
which distinguishes between objects, forwardly or rearwardly 

20 facing infant seats, and adult occupants by periodically 
mapping an image taken of the interior of the vehicle into 
image profile data, and utilizing image profile matching with 
stored reference profile data to determine the occupant or 
object type. Instantaneous distance is also measured and 

25 changes in the measured distances are tracked. All of this 
information is than used to optimize deployment control of at 
least one passenger safety restraint. 

Thus, in accordance with a first aspect of the 
present invention, a system for determining vehicle occupant 

30 type and position relative to a fixed structure within the 
vehicle comprises an imaging means mounted at a single location 
within the vehicle interior and having a predetermined field of 
view so that a front driver side seat and a front passenger 
side seat are both simultaneously viewable by said imaging 

35 means, but not simultaneously in focus as described more fully 
hereinbelow. The imaging means generates an output signal 
representative of an instantaneous position for any object 
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located within the field of view. The system further includes 
means for storing predetermined object profile data 
characteristic of a plurality of different types of objects 
when situated in either front seat of the vehicle, and a 
5 processor means for identifying the type of object located in 
the front seats of the vehicle by comparing the imaging means 
output signal to the predetermined object profile data. 

In accordance with a second aspect of the present 
invention, a method for determining position of an object 

10 located in a vehicle relative to a fixed structure within the 
vehicle comprises the steps of generating two-dimensional image 
data representative of any objects located within a front 
seating are of the vehicle, generating a two-dimensional range 
grid by vertically dividing the front seating area into a 

15 plurality of independent regions each representative of a 
predetermined size of the vehicle interior, wherein the fixed 
vehicle structure such as the instrument panel is located 
proximate to one end region, and the vehicle front seats are 
located proximate with the opposite end region, detecting 

20 lateral location of the object relative to a narrow depth of 
focus reference plane using a de-blurring filter, and 
determining distance from the fixed structure by comparing the 
generated image data with the range grid to detect which if any 
of the plurality of regions are occupied by an object. 

25 In achieving both of these aspects, the system and 

method of the present invention further comprise discriminating 
between objects and occupants; A-pillar positioning of the 
imaging system; use of a perspective angle correction lens; use 
of a two-dimensional range grid; tracking the change in 

30 instantaneous occupant position to predict a crash situation; 
optimization of passenger restraint deployment based on the 
identification of occupant type and position; generating the 
two-dimensional range grid by either etching the grid on a lens 
element, printing the grid on a CCD element, or utilizing 

35 suitable programming in a processor means; and estimating 
occupant lateral distance from a fixed vehicle interior 
component by utilizing a narrow depth of focus lens in 
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combination with suitable electromechanical or image processing 
auto- focus techniques. 

The present invention will be more fully understood 
upon reading the following detailed description of the 
5 preferred embodiment in conjunction with the accompanying 
drawings . 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a schematic illustration of a vehicle 
occupant type and position detection system in accordance with 
10 the present invention; 

Fig. 2 is a side view of the vehicle interior showing 
location of the system of the present invention; 

Fig. 3 is a downward view of the vehicle interior 
illustrating the field of view for the system of the present 
15 invention; 

Figs. 4 (a) and (b) are a side view of a "person 11 
type occupant and the corresponding two dimensional ranging 
grid image of the present invention; 

Figs. 5 (a) and (b) are a side view of an infant 
20 safety seat and the corresponding two dimensional ranging grid 
image of the present invention; and 

Fig. 6 is a flowchart illustrating the operation of 
the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT ( S) 
25 Referring to Fig. 1, there is shown a system 10 which 

determines vehicle occupant position and type in accordance 
with the present invention. In accordance with the preferred 
embodiment, system 10 is suitably adapted for mounting in a 
single location, such as the driver or passenger side A-pillar 
30 location as shown in Fig. 2, so as to have a field of view 
allowing the system to "see" any region within the driver side 
seat and the passenger side seat area for a given focal depth. 
The use of a single location advantageously reduces the amount 
of necessary hardware, and therefore the cost and complexity of 
35 manufacturing and installation of the present invention. 

More specifically, in accordance with the preferred 
embodiment, a single camera unit 12 is located at the A-pillar 
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or similar location so as to have a perspective field of view 
simultaneously covering both driver and passenger side seats. 
The camera unit 12 is preferably a low light infra red (IR) 
sensitive type camera system, and is arranged to provide all 
5 light operation through the inclusion of a supplemental light 
source, such as represented by an LED 14. However, one of 
ordinary skill in the art will readily appreciate that other 
types of camera systems may be suitable, and as such the use of 
an IR camera system 12 is not to be construed as limiting the 

10 present invention. 

A perspective angle correcting lens 16 is employed to 
translate the perspective image of the interior of the vehicle 
into a two-dimensional image signal which is then output to an 
image processor 18. In other words, lens 16 is optically 

15 designed to effectively remove the "perspectiveness" of the 
image created by the slant angle of the camera 12 with respect 
to a plane passing perpendicular to the instrument panel 
through the center of the passenger seat, as more clearly shown 
in Fig. 3. The correction lens 16 removes distortion in the 

20 image pixels inherently caused by the perspective view and 
makes the pixels all equal in actual distance spacing as if 
being two-dimensionally viewed from the side of the vehicle. 

In further accordance with the present invention, 
system 10 utilizes a de-blurring filter arrangement 19 in 

25 conjunction with a narrow depth of focus reference plane. With 
a narrow depth of focus, points along a perpendicular plane are 
in focus, and objects either farther or closer than this plane 
are out of focus. Thus, the location of an object relative to 
the focus plane can be inferred from the amount of blur in the 

30 image, i.e., the farther an object is laterally displaced from 
the focus plane, the more the image will be blurred. 

The de-blurring filter, while symbolically shown in 
Fig. 1, is preferably implemented as an algorithm subroutine in 
image processor 18. Therefore, with a narrow depth of focus 

35 and de-blurring filter arrangement, image processor 18 is able 
to infer or estimate the location of an object relative to the 
focus plane, while also being able to ignore or distinguish 
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background clutter within the viewable image, such as door 
features or the driver depending on which side of the car the 
system is located, from the desired image of the occupant or 
object within the seat. 
5 The system 10 further includes a grid pattern 20 that 

is either etched on the lens 16, screen printed on a CCD, or 
implemented by suitable programming within the image processor 
18 . This grid pattern is customized for each model of 
automobile, and as best illustrated in Figs 4 (a) -(b) and 5(a)- 

10 (b) provides an actual distance spacing metric for the pixels 
in the image along the plane perpendicular to the instrument 
panel. As described hereinbelow, the grid pattern 20 is the 
mechanism by which the image processor 18 will be able to 
. measure the actual distance of objects within the image focal 

15 plane, and to otherwise detect dominate features of an object 
or occupant located in the vehicle seat. The grid is 
effectively normalized by using lateral location derived from 
the narrow depth of focus reference plane and the de -blurring 
filter arrangement . 

20 In addition, the light source is preferably etched 

with a matching pattern of grid marks. The relative warping of 
the transmitted light by the occupant provides for detection of 
fine shape features which may not be otherwise discernable in 
poor lighting conditions due to reduced image contrast. Such 

25 a warped grid analysis also provides a three-dimensional 
profile of the occupant which can be used in conjunction with 
the detected lateral distance to provide comprehensive 
information regarding occupant size, shape, and location within 
a vehicle seat . 

30 The overall operation of the present invention as 

well as the remaining elements of Fig, 1 will now be discussed 
in context with the flow chart shown in Fig. 6. At step 100, 
the camera unit 12 generates a perspective image signal which 
is translated at step 102 by lens 16 into a two-dimensional 

35 image data signal representative of the profile of any objects 
located within a front seat of the vehicle. At step 104, a 
two-dimensional spacing or range grid is created by vertically 
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dividing the two-dimensional image of the front vehicle seating 
area into a plurality of independent regions each 
representative of a predetermined size of the vehicle interior. 
The grid is oriented so as to substantially locate the fixed 
5 vehicle structure at one end region, and the vehicle front 
seats substantially at the opposite end region, as particularly 
shown in Figs 4(b) and 5(b) . 

At step 106, the two-dimensionally generated image 
data is analyzed against the spacing grid to detect which if 

10 any of the plurality of regions are occupied by an object or 
occupant. The image processor 18 utilizes an image analysis 
algorithm which detects the dominant features and position of 
the object located in the vehicle seat at step 108. These 
features include the relative extent of the vertical portion of 

15 the occupant relative to the horizontal portion as shown in 
Figs. 4(b) or 5 (b) . 

Object profile data representative of a set of 
reference features of various types of occupants, such as 
humans, forward and rearward facing infant safety seats, or 

20 other inanimate objects, are stored in a suitable memory device 
21, such as a RAM or EEPROM. The stored sets of reference 
features are scaled to allow identification of a complete range 
of occupant sizes, i.e., small children to large adults. In 
addition, since there are a variety of different sized infant 

25 seats, a size invariant classification of reference features is 
provided for proper identification of infant seats. Image 
processor 18 than determines occupant type at step 112 by 
mapping or comparing the detected dominant features of the two 
dimensional image signal with the set of reference features 

30 stored in memory 21 at step 110. 

In addition to determining occupant type, the 
distance between the occupant and the instrument panel or 
steering wheel of the vehicle is detected at step 114 by 
measuring the relative location of the occupant based on the 

35 regions of the two-dimensional grid which are detected as being 
occupied at step 106. The actual distance is derived by 
combining the measured location within the grid with the 



-9- 



W09848372 [http://wvw.qetthepatent.com^ 12 of 22 



PCT/US9S/07685 



WO 98/48372 



lateral location of the object relative to narrow focus plane 
provided by the camera lens as determined by the de-blurring 
filter operation. 

The motion of the occupant is determined by looking 
5 for areas of relative motion and instantaneous distance changes 
through the grid zones, and to estimate the relative speed of 
the motion in these areas. Such information and the respective 
instantaneous changes in distance are stored, such as in memory 
21. At step 116, the data generated from this process is 

10 compared with contemporaneous vehicle speed data input on a 
line 22 from a centralized microprocessor air bag deployment 
control unit 24, or directly from one or more vehicle 
acceleration sensors (not shown) . This comparison step 
facilitates an analysis by either processor 18 or 24 of the 

15 motions of the occupants during either precrash or noncrash 
braking situations, which subsequently allows processor 24 to 
predict the onset of a crash and/or to develop an optimal 
deployment strategy for the air bags, or other restraints such 
as pretensioners and energy management systems. 

20 After determining occupant type and position at steps 

112 and 116 respectively, at step 118 the image processor 18 
provides an output signal 26 to the control unit 24, which 
subsequently optimizes control over the actuation, or 
deployment, of one or more passenger safety restraints, such as 

25 an air bag 28, or the activation of an audible or visual 
warning device (s) 30 via at least one output line 32. The 
warning devices 30 provide an alert for the vehicle occupant of 
a potentially hazardous seating condition. 

Therefore, with the present invention, the vehicle 

30 occupant type and position detection system 10 is designed to 
provide both high frequency detection of the type of occupant 
or object located in a vehicle, and measurements of the 
position of the driver and/or passengers relative to potential 
impact points such as the steering wheel and dashboard, and to 

35 process that information so as to provide an optimized safety 
restraint deployment decision. The system 10 thus allows the 
deployment control processor unit 24 to refrain from deploying 
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an air bag when a infant safety seat is present, particularly 
a rearward facing infant seat, or when a person is present but 
is too close, thereby preventing the explosive force with which 
an air bag is inflated from doing substantial harm to the 
5 infant or person. 

It will be further understood that the foregoing 
description of the preferred embodiment of the present 
invention is for illustrative purposes only, and that the 
various structural and operational features herein disclosed 
10 are susceptible to a number of modifications, none of which 
departs from the spirit and scope of the present invention as 
defined in the appended claims. 
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WE CLAIM: 



1. A system for determining vehicle occupant type and 
position relative to a fixed structure within the vehicle 
comprising: 

an imaging means mounted at a single location within 
5 the vehicle interior and having a predetermined 

field of view so that a front driver side seat 
and a front passenger side seat are both 
simultaneously viewable by said imaging means 
but not simultaneously in focus, wherein said 
10 imaging means generates an output signal 

representative of an instantaneous position of 
any object located within the field of view; 
a memory for storing predetermined object profile 
data characteristic of a plurality of different 
15 types of objects when situated in either front 

seat of the vehicle; and 
a processor means for identifying the type of object 
located in the front seats of the vehicle by 
comparing said imaging means output signal to 
20 said predetermined object profile data. 

2. The system of claim 1 wherein said processor means 
comprises means for determining the distance between an object 
located within the field of view and the fixed structure within 
the vehicle based on said imaging means output signal. 

3. The system of claim 2 further comprising means for 
optimizing deployment of a passenger safety restraint based on 
the identified type of object and distance. 

4 . The system of claim 3 further comprising means for 
storing instantaneous distance measurements, and means for 
tracking changes in the instantaneous distance measurements, 
wherein said means for optimizing safety restraint deployment 
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5 comprises means for predicting a vehicle crash based on said 

tracked changes in distance. 

5. The system of claim 2 further comprising: 
means for generating two-dimensional image data 

representative of any objects located within 
the predetermined field of view; and 
means for generating a two-dimensional range grid 
which vertically divides the field of view into 
a plurality of independent regions each 
representative of a predetermined size of 
vehicle interior space, wherein said grid is 
oriented so that the fixed vehicle structure is 
located proximate with one end region, and the 
vehicle front seats are located proximate with 
the opposite end region, wherein said means for 
determining distance comprises means for 
comparing said generated two-dimensional image 
data with said range grid to detect which if 
any of the plurality of regions are occupied by 
an object. 

6 . The system of claim 5 wherein said imaging means 
comprises a lens element, and said means for generating a two- 
dimensional range grid comprises a grid pattern etched on the 
surface of said lens element . 

7. The system of claim 6 wherein said imaging means 
further comprises means for transmitting light into the 
predetermined field of view, and said means for generating a 
two-dimensional range grid further comprises a grid pattern 

5 etched onto said light transmitting means which matches said 

grid pattern etched onto said lens element. 

8 . The system of claim 5 wherein said imaging means 
comprises a CCD element, and said means for generating a two- 
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dimensional range grid comprises a grid printed on said CCD 
element. 

9. The system of claim 8 wherein said imaging means 
further comprises means for transmitting light into the 
predetermined field of view, and said means for generating a 
two-dimensional range grid further comprises a grid pattern 

5 etched onto said light transmitting means which matches said 

grid pattern etched onto said CCD element. 

10. The system of claim 5 further comprising a narrow 
depth of focus lens coupled to said imaging means and a de- 
blurring filter coupled to said processor means, wherein said 
means for determining the distance is responsive to said narrow 

5 depth of focus lens, said de-blurring filter, and said two- 

dimensional grid for estimating the absolute distance based on 
the lateral location of the object within the field of view 
relative to the fixed structure of the vehicle. 

11. The system of claim 1 wherein said single location 
comprises a vehicle A-pillar, and wherein said imaging means 
further comprises a perspective angle correction lens for 
translating the predetermined field of view from a perspective 

5 view to a two-dimensional view. 

12. The system of claim 1 further comprising means for 
optimizing deployment of a passenger safety restraint based on 
the identified type of object. 

13. The system of claim 5 wherein said means for 
generating two-dimensional image data comprises a processor 
having a deblurring algorithm. 

14. A method for determining position of an object 
located in a vehicle relative to a fixed structure within the 
vehicle comprising the steps of: 
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generating two-dimensional image data representative 
of any objects located within a front seating 
are of the vehicle; 
generating a two-dimensional range grid by 
vertically dividing the front seating area into 
a plurality of independent regions each 
representative of predetermined size of vehicle 
interior, wherein the fixed vehicle structure 
is located proximate with one end region, and 
the vehicle front seats are located proximate 
with the opposite end region/ and 
detecting lateral location of the object relative to 
a narrow depth of focus reference plane using a 
de-blurring filter; and 
estimating the distance between the object and said 
fixed structure by comparing said generated 
image data with said range grid and detected 
lateral location to determine which if any of 
the plurality of regions are occupied by an 
object. 

15. The method of claim 14 wherein said step of 
generating a two-dimensional range grid comprises etching said 
grid onto the surface of a lens element used to generate said 
two-dimensional image data. 

16. The method of claim 14 wherein said step of 
generating a two-dimensional range grid comprises the step of 
printing said grid on a CCD element used to generate said two- 
dimensional image data. 

17. The method of claim 14 wherein said step of 
generating a two-dimensional range grid comprises etching 
matching grid patterns onto each of a light transmitting and 
receiving component used to generate said two-dimensional image 

5 data, and further comprising the step of estimating a profile 

of any object located within the front seating area by 



10 



15 
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detecting warping of the transmitted light grid relative to the 
receiving light grid. 

18. The method of claim 14 further comprising the steps 

Of: 

storing predetermined object profile data 
characteristic of a plurality of different 
5 types of objects when situated in either front 

seat of the vehicle; and 
identifying the type of object located in the front 
seats of the vehicle by comparing said two- 
dimensional image data to said predetermined 
10 object profile data. 

19. The method of claim 18 further comprising the step of 
optimizing deployment of a passenger safety restraint based on 
the identified type of object and distance. 

20. The method of claim 19 further comprising the steps 
of storing instantaneous distance measurements, and tracking 
changes in the instantaneous distance measurements, wherein 
said optimizing step comprises the step of predicting a vehicle 

5 crash based on said tracked changes in distance. 

21. The method of claim 14 wherein said step of 
generating two-dimensional image data comprises the step of 
positioning an imaging system having a predetermined field of 
view at a single location within the vehicle. 

5 

22. The method of claim 21 wherein said single location 
comprises a vehicle A-pillar, and wherein said step of 
generating two-dimensional image data further comprises the 
step of translating the predetermined field of view from a 

5 perspective view to a two-dimensional view. 
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